Experimental Evaluation of Auditory Display and Sonification of Textured Images

In order to verify the potential of proposed auditory display and sonification methods for aural analysis of textured images, a set of experiments was designed and was presented to 10 subjects. The results obtained and limitations of the methods are discussed. INTRODUCTION It has been shown that the auditory system is very useful for task monitoring and analysis of multidimensional data [1]. However, the use of sound in scientific data analysis is rather rare, and analysis and presentation of data are done almost exclusively by visual means. Even when data are the result of sounds, such as an ultrasound exam or sonar, they are first mapped to an image and visual analysis is performed. Sonification of data can be generally represented as in Figure 1. Data characteristics need to be mapped to sound attributes in order to be presented and analyzed. This mapping is an important task, and appropriate design of the mapping function is the difference between a successful sonification procedure or failure. Figure 1 Sonification process. Many processes related to our everyday experiences can be used to map data to sound, such as metaphorical and affective association [2]. Common metaphorical associations include high frequency for an increase in a data characteristic or a rapidly changing parameter of the data, and conversely, low frequency for a decrease or slow rate of change. Affective association tries to associate the feelings evoked by a sound with the data controlling the sound. As an example, in a project on the identification of features in magnetic resonance images of the brain, an initial suggestion from our collaborators in the field of radiology was to map pleasant sounds to healthy tissue and unpleasant sounds to diseased areas. A few mappings from data to sound have been established in the literature [1]. Walker and Kramer [3] argued that one of the considerations that should be taken into account is that a particular mapping choice must rely on the performance of a task, rather than on merely the feeling of the designers that the choice made is intuitive; their preliminary result shows that what is intuitive for some may not be intuitive for others. We are particularly interested in sonification of textured images [4]. Among the many problems involved in this task, the first that comes to mind is that sound is essentially a change of pressure with time, i.e., sound is something that evolves in time, whereas an image is a static object. How can something static be mapped into something that is time-varying? One way of addressing this problem is to choose a sound signal, for example a sinusoidal oscillator, and associate pixel mapping of data characteristics to sound attributes input data sound